Visual Target Recognition and Tracking for Autonomous Manipulation Tasks

Increased levels of robotic system autonomy will enable scientific exploration in previously unreachable destinations. Dexterous manipulator systems can be coupled with machine vision systems to increase perceptive capabilities and provide greater opportunities for scientific return through environmental interaction. Reliable autonomous vision systems are often specific to a certain task and quickly become complex as new operational tasks are added. This paper describes the development of a flexible stereovision system coupled with the University of Maryland’s Ranger manipulator system to identify, track, and sample arbitrary targets within the manipulator’s workspace. The vision system is designed to be deployed on an autonomous underwater vehicle (AUV) that will conduct deep-sea sampling missions autonomously, where lighting and visibility constraints are formidable. Laboratory testing demonstrates the vision system’s ability to repeatedly and reliably select desired targets and provide target position data to the manipulator during autonomous sampling tasks. Visual servoing results demonstrate closed-loop tracking of static and moving targets.